Advances in Speech Recognition Using Sparse Bayesian Methods

The prominent modeling technique for speech recognition today is the hidden Markov model with Gaussian emission densities. They have suffered, though, from an inability to learn discriminative information and are prone to overfitting and overparameterization. Recent work on machine learning has moved toward models such as the support vector machine that automatically control generalization and parameterization as part of the overall optimization process. The support vector machine, however, requires ad hoc (and unreliable) methods to couple it to probabilistic speech recognition systems. In this work, we introduce the use of a probabilistic Bayesian learning machine termed the relevance vector machine as the core pattern recognition unit in a speech recognizer. The relevance vector machine system is compared to previous work using support vector machines and is found to outperform the support vector machine system in terms of both accuracy and sparsity on a continuous alphadigit task.